409 research outputs found
Cooperative Convex Optimization in Networked Systems: Augmented Lagrangian Algorithms with Directed Gossip Communication
We study distributed optimization in networked systems, where nodes cooperate
to find the optimal quantity of common interest, x=x^\star. The objective
function of the corresponding optimization problem is the sum of private (known
only by a node,) convex, nodes' objectives and each node imposes a private
convex constraint on the allowed values of x. We solve this problem for generic
connected network topologies with asymmetric random link failures with a novel
distributed, decentralized algorithm. We refer to this algorithm as AL-G
(augmented Lagrangian gossiping,) and to its variants as AL-MG (augmented
Lagrangian multi neighbor gossiping) and AL-BG (augmented Lagrangian broadcast
gossiping.) The AL-G algorithm is based on the augmented Lagrangian dual
function. Dual variables are updated by the standard method of multipliers, at
a slow time scale. To update the primal variables, we propose a novel,
Gauss-Seidel type, randomized algorithm, at a fast time scale. AL-G uses
unidirectional gossip communication, only between immediate neighbors in the
network and is resilient to random link failures. For networks with reliable
communication (i.e., no failures,) the simplified, AL-BG (augmented Lagrangian
broadcast gossiping) algorithm reduces communication, computation and data
storage cost. We prove convergence for all proposed algorithms and demonstrate
by simulations the effectiveness on two applications: l_1-regularized logistic
regression for classification and cooperative spectrum sensing for cognitive
radio networks.Comment: 28 pages, journal; revise
Large Deviations Performance of Consensus+Innovations Distributed Detection with Non-Gaussian Observations
We establish the large deviations asymptotic performance (error exponent) of
consensus+innovations distributed detection over random networks with generic
(non-Gaussian) sensor observations. At each time instant, sensors 1) combine
theirs with the decision variables of their neighbors (consensus) and 2)
assimilate their new observations (innovations). This paper shows for general
non-Gaussian distributions that consensus+innovations distributed detection
exhibits a phase transition behavior with respect to the network degree of
connectivity. Above a threshold, distributed is as good as centralized, with
the same optimal asymptotic detection performance, but, below the threshold,
distributed detection is suboptimal with respect to centralized detection. We
determine this threshold and quantify the performance loss below threshold.
Finally, we show the dependence of the threshold and performance on the
distribution of the observations: distributed detectors over the same random
network, but with different observations' distributions, for example, Gaussian,
Laplace, or quantized, may have different asymptotic performance, even when the
corresponding centralized detectors have the same asymptotic performance.Comment: 30 pages, journal, submitted Nov 17, 2011; revised Apr 3, 201
Distributed Detection over Random Networks: Large Deviations Performance Analysis
We study the large deviations performance, i.e., the exponential decay rate
of the error probability, of distributed detection algorithms over random
networks. At each time step each sensor: 1) averages its decision variable
with the neighbors' decision variables; and 2) accounts on-the-fly for its new
observation. We show that distributed detection exhibits a "phase change"
behavior. When the rate of network information flow (the speed of averaging) is
above a threshold, then distributed detection is asymptotically equivalent to
the optimal centralized detection, i.e., the exponential decay rate of the
error probability for distributed detection equals the Chernoff information.
When the rate of information flow is below a threshold, distributed detection
achieves only a fraction of the Chernoff information rate; we quantify this
achievable rate as a function of the network rate of information flow.
Simulation examples demonstrate our theoretical findings on the behavior of
distributed detection over random networks.Comment: 30 pages, journal, submitted on December 3rd, 201
Distributed Detection over Noisy Networks: Large Deviations Analysis
We study the large deviations performance of consensus+innovations
distributed detection over noisy networks, where sensors at a time step k
cooperate with immediate neighbors (consensus) and assimilate their new
observations (innovation.) We show that, even under noisy communication,
\emph{all sensors} can achieve exponential decay e^{-k C_{\mathrm{dis}}} of the
detection error probability, even when certain (or most) sensors cannot detect
the event of interest in isolation. We achieve this by designing a single time
scale stochastic approximation type distributed detector with the optimal
weight sequence {\alpha_k}, by which sensors weigh their neighbors' messages.
The optimal design of {\alpha_k} balances the opposing effects of communication
noise and information flow from neighbors: larger, slowly decaying \alpha_k
improves information flow but injects more communication noise. Further, we
quantify the best achievable C_{\mathrm{dis}} as a function of the sensing
signal and noise, communication noise, and network connectivity. Finally, we
find a threshold on the communication noise power below which a sensor that can
detect the event in isolation still improves its detection by cooperation
through noisy links.Comment: 30 pages, journal, submitted August 2nd, 201
- …